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Preface 


This report describes the testing activities which 
have been perfoirmed as a part of the HAL/S-360 compiler 
development project. 

There have been eight HAL/S— 360 compiler releases, 
designated 360-1 through 360-8. 360-1 and 360-2 were 

early developmental in-house versions of the compiler. 

They were used for design purposes, and performance 
was verified largely by visual inspection of produced 
code. No real execution was planned*. 

Releases 360-3 through 360-8 were compilers with 
successively greater capabilities. These compilers were 
issued to the Shuttle programming community (viz. Rockwell, 
Draper, IBM/Houston, Honeywell). Discrepancy reporting 
began with 360-3**. Formal testing activities, according 
to plan, have been conducted on all releases from 360-3 
forward. Plans and results have been recorded by 
Intermetrics Shuttle Information Exchange (SIE) Memos. 

This current publication summarizes all of the 360 test 
activities and is meant to serve as' a final test report. 


* Results were described in Intermetrics reports issued 
on 13 April 1973 and 15 June 1973. 

** This release was also described by Intermetrics report 
issued on 28 August 1973. 
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1.0 » INTRODUCTION: LEVELS OF TESTING 


The testing of HAL/S can be undertaken at several 
different levels. At the highest level of abstraction, 
the usefulness of HAL/S as a programming tool can be 
tested by using it in typical applications programming 
cases. Such testing has been and continues to be an 
important source of changes and modifications to the 
HAL/S Language Specification itself, as well as serving 
as a compiler-verification tool (see Section 3.1). 

At a lower level, given the current language defini- 
tion, the functional characteristics of the compiler system 
can be tested- Here the object is to verify that the 
compiler produces object code which is a faithful transla- 
tion of the original HAL/S Program source. This functional 
testing of the compiler and its generated code is performed 
by a judicious selection of significant test cases which 
are compiled and then executed. Selection criteria for 
tests are based on many inputs, ranging from knowledge of 
the compiler's internal structure to the known "good" 
results of mathematical built-in functions. 

At the lowest level there is the machine oriented 
testing of a particular compiler's implementation - such as 
the H7Ui/S-360 implementation on the IBM 360 or the HAL/S— FC 
implementation for the AP— 101 flight computer. Here the 
emphasis is placed on the details of the resulting object 
code, e.g. are internal register assignments correct? is 
optimization working properly? etc. In a great many 
instances, problems revealed at the functional level provide 
indicators of significant areas to test at this level. 

All three levels of testing are employed in verifying 
the HAL/S-360 compiler. 
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2.0 INITIAL TEST PLAN 

The formal program of testing the HAL/S Compiler 
for the System 360 began just prior to the first 
operational release of the HAL/S compiler, Release 
360-3 (August 1973) . Subsequent evolution of the 
language itself, as well as the continued work on 
testing, have together resulted in modifications and 
improvements of the test plan. 

This section contains a description of the HAL/S— 360 
test plan as originally formulated. Section 3, which 
follows, contains information on additional features in- 
corporated in the test plan during the course of compiler 
development. 


2 , 1 Test Philosophy and Test Procedure 

If it is assumed that the definition of a working 
compiler is one which produces correct object code for 
every possible legal input, then clearly, the primary 
criterion for a compiler test plan is that it examine 
the code produced for every possible source statement. 

However, to take this requirement literally would produce 
a test program with a hopelessly large number of statements; 
the number of syntactic forms in the language factorial, 
might be a good first approximation. If this were necessary, 
it would not be practical to test the compiler at all. A 
more efficient method of testing than this "black box" 
approach must be utilized. 

The saving feature is that of the huge number of possible 
HAL/S statements, most are different only to the programmer 
and not to the compiler. For instance, the "black box" or 
outside approach dictates that the addition operator, *+* , 
must be tested with all possible combinations of positive 
and negative operands. This is really not necessary, as 
examination of the compiler will show that the negative 
operands, once converted to an internal form, are treated 
exactly the same as-positive operands. In fact, the sign 
of an operand is not usually known by the compiler. Thus, 
an "inside" approach to testing may be developed. It is 
necessary only to test the several cases of this conver- 
sion to internal machine representation, and the test of 
the addition operator is reduced by a factor of four, or more. 
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This is not an isolated case; in fact, it is the rule rather 
than the exception. Testing based on a detailed knowledge 
of the compiler’s inner workings eliminates the gross re- 
dundancy of the outside, or user, approach. Thus, the test 
philosophy becomes one of exhaustively testing all meaning- 
fully different inputs to the compiler, where a "meaningful” 
difference is defined as one that is preserved through the 
initial passes of the compiler, and causes a different path 
to be taken in generating object code. This approach may 
require more time on the design of test cases, but it reduces 
these cases to a tractable number and further, makes it 
possible to examine the output code for correctness, in 
addition to comparing the results of the execution of that 
code against a predetermined answer. 


The general problem of testing the HAL/S-360 compiler can 
be separated into several categories. The categories are chosen 
through an analysis of the structure of the compiler, 
paralleling its logically disjoint sections. Four major 
headings are identified (see Table 1) : 

a. the hand-coded portions of the first phase of the 
compiler which are listed under PASS 1; 

b. those language features listed under Discrete which 
are more basic and self-contained; 

c. those under General which have wider implications 
and which will require more special testing; and 

d. System Support Features which cotains those categories 
which facilitate communication with the environment, 
and simpligy the writing of programs. 

Certain language features, such as declarations, are not 
listed, but are implicitly tested by their use in the test 
routines for all categories. These implicit tests, together 
with the categories listed in Table 1, form a comprehensive 
set of language elements which must be tested. Based on this 
organization, the following procedure will be followed to 
produce all of the necessary tests. 
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3 l\ Select a category. 

b. Determine sub-categories. 

c. Classify sub— categories into enumerable vs. special. 

d. Produce test matrices for each enumerable sub-category. 

e. Generate tests from these matrices. 

f. Generate special case tests. 

Within each category, sub-catcgories will be produced by a 
process similar to that which generated the category itself. 

^hese sub-categories tend to split along the lines of statement 
types, rather than along the lines of language features, as the 
initial categories did. This type of classification scheme, 
while somewhat redundant, simplifies the testing process, and ^ 
increases the probability that no language construct will remain 

untested. 

Next, the sub-categories will be divided into two groups: 
enumerable and special. The first group will test those parts 
of the compiler through which several paths may be taken, 
depending upon the context in which they are used. For instance, 
the exponentiation routine’s behavior varies considerably 
depending on the arguments it is called with. An example of 
a "special" sub-category is the test of the multiple assignment 
processor. This routine always functions in the same way, 
seguencing through the receiving fields on the left of the 
assignment statement, evaluating each one, and calling some 
other routine to do the assignment. The routine that actually 
does the assignment is tested as one of the _ "enumerable" 
sub-categories, as it must do a conversion if the data types 
involved are not identical . 

The tests required for one of these "enumerable sub- 
categories are represented by a matrix. The matrices for 
the Integer-Scalar sub-categories are listed in Appendix A. 

Refer to the matrix A3: This matrix is accompanied by the 

expression <column element>**<row element> . The X in the matrix 
which has been circled indicates that one of the test cases 
must be a double precision scalar raised to the power of a 
double precision scalar. In general, an X in the matrix 
indicates that the label above the X (signified by 
<column element>) must be raised to the power of the label 
to the left of it (signified by <row element>) . In one of 
these matrices, an X may be omitted if and only if there is 
already a case which will take the same path through compilation* 
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TABLE 1 


Testing Categories 


Pass 1 Discrete 

error checks 
output writer 
scanner 


General 

library 
built-ins 
Array ness 

structure copyness 
relational expressions 
arrayed relations 
procedure link and parameter 
pass 

addressability 
register usage 
update , task blocks 
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System Support Features 

macros 

includes 

PDS manipulation 

files 

compool 

comsubs 

locks 

real time 


Integer, Scalar 
Vector, Matrix 
Bit 

Character 
Structure 
Flow Control 
Conversions 
I/O 



Finally, test cases must be generated for the "special" 
sub-categories. Here the matrix formalism is not applicable. 
These special cases must be made to trigger specific pathways 
through the compiler, and their design requires a detailed 
knowledge of the decision points along the compilation process. 
Wherever a decision point occurs in the compiler , a test case 
must be prepared to follow each branch. The results of these 
special test cases must be checked by an examination of the 
object code rather than by checking the results of execution, 
but this is advantageous in that the number of test cases may 
be reduced by combining several tests into one statement. 


2 . 2 EXAMPLE : INTEGER-SCALAR 


To illustrate the testing strategy, one of the major cate- 
gories from Table 1 will be explained in detail down to actual 
test cases. The example that has been chosen is Integer-Scalar. 
The primary reason for selecting this category for first 
consideration is that it will be so v/idely used in IIAL/S 
programs 7 an error here is likely to cause errors in the majority 
of compiled programs. In addition, an early test of this^ 
category may show up widespread errors in the compiler , since 
the testing of Integer-Scalar operations must involve the use 
of many other language features. Finally, the Integer-Scalar 
category is large enough to illustrate many of the problems 
encountered in the test procedure and point to their solutions. 

To simplify the task, the Integer-Scalar category was 
divided into several sub-categories along the lines of 
statement type (see Table 2) , These sub— categories overlap 
to cover all uses of integers that are not more appropriately 
covered under one of the other categories listed under General 
in Table 1 . 

Appendix A contains appropriate matrices for the enumerable 
sub-categories of Integer-Scalar. Some matrices indicate many 
possibilities where as in others only a smaller number of cases 
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' TABLE 2. 

Categories Tested Under Integer, Scalar* 


Category Matrix 

1. Assignments X 

2. Initial Values X 

3. Comparisons X 

4^ Binary Mathematical 

Operators X 

5. Exponentiation X 

6^ • Unary Operators X 


7- Multiple Assignments 
8, Complex Expressions 


9, Complex Comparisons 


Comments 


All tested due to implicit 
conversions. 


Not including ** (exponen- 
tiation) 

Separate category due to 
required internal handling 
by compiler. 

+ and = prefixes. 

Test .of ordering of assign- 
ment included. 

Tests of precedence; elementary 
function, returns, definitions, 
and element selection implicitly 
tested ♦ 

Tests of precedence , combina- 
tions with BOOLEAN variables 
included . 


* Note: Certain declares and initialization, (simple I/O and 

flow coiTtrOl , comment ability, and no argument 
procedures call) implicitly tested. 
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will need to be tested. Those cases which are eliminated are 
done ‘so due to certain properties of the compiler, primarily 
commutivity. For example, under the operators ' + ' , and 

V' / the tests form a triangular matrix due to the "bubbling 
up" (to higher precision) property of the compiler; a scalar 
of double (SD) precision divided by a scalar of single precision 
(SS) is essentially the same as SS/SD since in both cases the 
SS is converted to an SD and then the division is performed. 

On the other hand, all possible assignments are tested due to 
implicit conversions. Each matrix will have notes explaining 
deletions of tests. 


2.3 Test Result Criteria 


Once the test programs have been run, the results must be 
verified. This can be done on two levels. The higher level 
is that of checking produced values vs. expected values. 

(This level is, in fact, the more important of the two in terms 
of correctness.) For two reasons the checkout of the HAL 
compiler will involve both levels. The first is that of 
completeness through retention of an outside check. The second 
is that in order to remain compatible with our inside approach, 
the object deck produced from a HAL program should be examined. 
Therefore, in a HAL statement such as A=5 ; where A is a scalar 
or single precision, one would check that A is indeed equal to 5 
and that the literal has been converted to a single precision scalar . 

It may be noted that the majority of test cases are simple 
rather than compound statements and expressions. The motiva- 
tion behind this is that intermediate results in the evaluation 
of expressions are usually kept in registers. Thus, they all 
have the same format, which means that there is no need to mix 
data types in the complicated expressions, after each combina- 
tion has been paired and checked in the simple expressions. 

More complex expressions in the Integer-Scalar category examine 
such things as order of evaluation of sub-expressions and the 
correct use of temporaries. Evaluation of these tests will be 
done by examination of the object code produced rather than by 
checking the results. Thus, as expressions get more complex, 
we move from examijiation of the results of many cases, to a 
check of the generality of the code produced for a few cases. 
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3.0 TEST PLAN ADAPTATION 

The original test plan, as described in Section 2 above, 
has been adapted and improved during the course of the HAL/S- 
360 testing activities. At the time of the final compiler 
version, Release 360-8, the number of test programs, for 
instance, had grovm to over 100 from an original count of 
40 programs. Language design features have also changed 
during the course of compiler development, and with each 
release the test cases in the library of test programs 
have been reviewed and modified to reflect such additions. 
This section describes the extensions to the original test 
plan which have been incorporated as of the date of this 
report. 


3. 1 Design Tests Incorporated into Formal Testing 

In carrying out the HAL/S— 360 test plan. Intermetrics 
took advantage of its knowledge of the internal design 
and implementation of the compiler. Special tests in order 
to verify specific constructs and/or algorithms were added 
to the set of Formal Tests. Some examples are described in 
this paragraph: 

1. Since the compiler’s replace mechanism sees the replace 
text as an uninterpreted character string, significant 
tests involve replace texts of varying length, nested 
replaces and parameterized replaces. 

2. Referencing elements of aggregate data types must be 
tested in cases where the total size of the data in 
question is greater than 4096 bytes (on the 360) . 

This is due to the fact that in some cases a subscript 
expression can be incorporated into the displacement 
field (12 bits) of a machine instruction. 

For a siinllar reason, HAL statements which increment 
an integer variable by more than 4096 must be tested: 

**I = I + 5;" can be done by a LA instruction 
(leaving the result in a register) whereas 
"1=1+ 5000;" cannot. 
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3. Si.nc 0 th© compil 02 r stores nuiltidinensional arrays 

(and matrices) in row-major order, tests of the form 
A$(*,3) = are necessary to verify assignments into 

array partitions which are not contiguous. 

i' 

4. The order of evaluation of parts of a statement must 
be tested in several cases other than expression 
context. An example is: 

"A$I,I = I+A$3". 

5. Since the compiler maintains a separate stack segment 
for each code block, assignment statements across task, 
procedure, and program boundaries must be used to verify 
correct accessina of data at various nest levels. 

6. Some computations are done at compile time. Thus, a 
complete test of exponentiation must include both 

"I = 3**12;" and "J = 3; I = J**12;". Tests of 
precedence and certain built-in functions must also be 
made in this context. 

7. Consider the following example: 

T: program; 

Declare integer initial (5) , A,C; 

Declare B bit (16) initial (Hex'8005'); 

A = abs (integer (b) ) ; 

C = abs (A) ; 

Write (6) A,C; 

Close; 

One might expect that Test 2 was the same as Test 1 
in the above example, since both statements take the 
absolute value of a single (default) precision integer. 
This, however, is not the case, due to the implementa- 
tion of the abs function and the peculariaties of the 
360 STH and LPR instructions. 


/*Test 1*/ 
/*Test 2*/ 
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3.2 Total Testing of the HAL/S-360 Compiler - Use Testing 

The programs which represent the implementation of 
the HAL/S-360 test plan are only a portion of the test 
effort that is in progress. All tests can be included 
in one of the following four classes; 

1. Formal Testing; The tests which are generated according 
to a test plan, 

2. Developmental Testing; The day to day test programs 
which are used to verify parts of the compiler as t ey 
are written . 

3. Visual Verification: The examination of the code as it 

is written to find special cases which have not been 
accounted for, 

4. Compiler Use: Programs which are written in HAL/S to 

be used in the course of other work. 

With respect to the last class ^ it should be noted that ^ 
certain types of errors are not easily caught by a systematic 
test plan. These errors can be found only when a progr^lm xs^ 
presented with one particular set of inputs. Therefore, it is 
useful to spot check the compiler randomly by subjecting xt to 
a number of application and utility programs. Such programs 
have been designed and are listed under Category 6 xn the 
tables of Appendix C. 

The uses of these programs range from calculations to 
debugging, to the generation of other test programs. Many 
of them were run on a daily basis. 


This type of program has been responsible for the 
detection of several errors during the course of the HAL/ S 
development effort. For instance, in release 3 of the compxler, 
an application program called QUEUES detected one o 
several discrep^cies in compiler operation when a partxcular 
combination of name choice and block nesting was compxled. 

In release 6 of the compiler, the programs KIP and FDA were 
responsible for the detection of an error triggered by the 
specific program swapping techniques employed in the real 
time aspects of these applications. These are not the only 
examples of such error detection. The technxque wxll contxnue 
to prove valuable in the future releases of HAL/S compxlers 
for the flight computer and the 360, if necessary. 
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3.2.1 PMF and FDI Simulators 

Two major HAL/S programming efforts were developed 
. at Intermetrics in support of Rockwell International, viz. 
a Performance Monitoring Function simulator and a Faxlure 
Detection Indication simulator. The HAL/S- 360 development 
has benefitted from this in-house use of the language and 
compiler. The programs are distinctly different and exercise 
almost all aspects of HAL/S from mathematics and structures 
to systems use and real time. They were developed in parallel 
with HAL/S-360 and were run successfully using several releases 
of the compiler. Several HAL/S discrepancies were discovered 
using these programs on developmental versions of the compiler. 


3. 2. 1.1 PMF . The Performance Monitoring Function simulation 
currently comprises ten external procedures, two Compools, 
and four programs. The external procedures are display 
handling primitives which are called by the following 
programs : 

1) Opei'ational Programs 

a) FDA. Executes all Fault Detection and Annuncia^ 
tion functions. Provides data base for SMM, 

SCM display functions. 

b) KIP. (Keyboard Input Processor) Control center 
for display processing. Responds to KBD 
inputs, and maintains control of the -CRT level 
usage by each of the four DEU*s. 

2) Simulation Support Programs 

a) ^ CREWMAN. Prepares simulated crew KBD inputs, 

and passes them to KIP, Also processes requests 
for CRT page print-outs, and provides general 
simulation control. 

b) EXEC. This is a small routine that does the 
initial scheduling of all the component programs, 
to get the simulation going. 

The combined programs and procedures contain approximately 
IfiOO IIAL/S statements and the Coinpool data occupies approximately 
10,000 360 v.’ords. The four programs contain eight task blocks 
and over tv.’enty internal procedures and functions. 
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reproducibility of raE 

. ^ T-» A /~iTTi TC T5r\OT?. 


3.2.1. 2 FDI, The FDI simulator is designed to evaluate 
sensor FDI algorithms under realistic Shuttle flight 
considerations. The following Program Layout taken from 
the compilation indicates the organization and scope of 
the simulation. 


PRCGRA*V layout 


J STATEUPDATE : EXTERNAL PROCIDURt; 


S'e'LFCT STATE' : cXTERNAL PRCCcDURE; 


RFiLnATA; EXTERNAL CCVPCCL; 

j| CRTHC: EXTERrjAL FUMCTICN; 

CCVPCATA; EXTERNAL CCPPOOL; 

^ _OUTPUTOATA: EXTERNAL PROCEDURE; 

J VEMCLrOYNA'UCS: EXTERNAL PROCEDURE; 

I 

’ imumocel: external PHCCEDUPE; 

NAVStNSORS: EXTERNAL PROCEDURE; 

ECU RE; 

“.'“NAVF ILTER: EXTERNAL PPOCEDURE; 
GUIDANCE: EXTERNAL FRCCEDURE; 


IMUPDI: EXTERNAL PKCCEI 


SHUTTLE^FniSIi^: PROGRAM; 
1 Basic OAT A: PROCEDURE; 


no 


II 


>2 


C_ASEOATA: PRQCcOURE; 
REALWOPLC; PROCEDUKE; 
’ CUM p'UTt: R : P ROC E OU R E ; ' 
T S T P ; PROCEDURE; 


The program contains over 14 0 0 II7iL/S state'ments and also 
includes 40 internal procedures and functions. 
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3.2.2 Boost Trajectory Generator 

This program simulates a boost-into-orbit trajectory. 
It is typical of guidance and control programming using all 
the mathematical features (integer, scalar, vector, matrix) 
available in HAL/S as well as arrays for formulating and 
manipulating tables of data. 


3.2,3 Other Testing 

Testing of the HAL/S compiler releases was also 
conducted at three other contractor installations : 

Draper Laboratory 

IBM/Houston 

Rockwell/Downey 

These contractors reported discrepancies (Discrepancy 
Reports) to NASA/JSC and Intermetrics, All reports were 
evaluated by Intermetrics and errors fixed in subsequent 
releases. 


3, 3 Mathematical Function Precision Testing 


The incorporation of precision tests into the testing 
of HAL/S-360 mathematical function routines was begun with 
Release 8 of the HAL/S 360 compiler. The purpose of this 
testing is to verify the mathematical function algorithms 
as implemented, for internal consistency and for consistency 
with known values of the functions. 

The initial approach to this form of testing was to 
analyze the algorithms used, looking for likely sources of 
error in the calculation and for input ranges where large 
errors can be expected. The purpose of this analysis was 
to find a set of worst case test points for each function 
which would then be compared with the exact mathematical 
values of the functions. For example, it was expected 
that the errors would be larger near singularities. There 
was some success with the method as applied for example to 
the Sine routine, testing multiples of tt for an exact zero 
value, thus analyzing the results of the finite precision 
of the constant tt used in making the principal angle 
corrections. This type of testing is continuing with more 
results to be expected at a later date. 
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Following the initial testing of mathematical 
routines as outlined above, it was decided to check 

the accuracy of the approach were the largest 

relative errors at the predicted points based on analysis 
of the algorithms? To perform this consistency checking, 
it was decided to use the double precision versions of ^ 
routines as the "standard” for comparing single precision 
results and calculating relative error as a function of 
the input argument range. Preliminary results indicate 
that for ranges of the argument which result in errors on 
the order of magnitude of the last digit, the error is not 
related to the "most likely" points determined by analysis. 

The source of this error is attributed to truncation as 
opposed to fundamental characteristics of the routine itself. 
For larger errors, the programs doing the testing were modified 
to isolate the regions of larger error and examine such 
regions with a finer granularity. Using this approach, it 
was found, for example, that the HAL/S single precision TANGENT 
function yields regions of arguments with errors three 
orders of magnitude larger than the largest IBM FORTRAN 
relative error in the equivalent routine, indicating a 
probable error in the HAL/S runtime routines. In order to 
extend this approach to double precision routines, it is 
necessary to obtain comparison standards for triple 
precision versions of the same functions. Work is now 
progressing on the definition of such extended precision 
algorithms for use in checking out the HAL/S double precision 
routines . 

All the function range testing being applied on the 360 
version of the compiler can and will be carried over to the 
flight computer compiler. This work can be accomplished due 
to the identical floating point data representations and the 
fact that identical algorithms (although differing in machine 
representation) are used for both computers. 


\ 
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4.0 TEST RESULTS SUMMARIES 


The HAL/S-360 testing activities have been carried 
out for each release of the HAL/S compiler. This section 
summarizes the results of tests, by release. 


4 . 1 Formal Testing of Release 360~3 

The test plans, as described above, were first applied 
to Release 360-7. A set of test categories and sub-categories 
was generated for this purpose. Test programs were written 
and exercised on 360-3 covering all features expressed in 
the plan. In the tables of Appendix C, the categories (and 
sub-categories) and the corresponding test programs (in caps) 
are indicated. (Appendix C shows the current list of programs) . 

In all, forty (40) test programs were written for 
categories A, B, C, D. These programs contained approximately 
4,000 individual logic and language feature tests for Release 
360-3. The bulk of source text, object code, and test 
results printout precluded their publication for distribution, 
however, these volumes are on file at Intermetrics, i.e: 

1. The original source program 

2. The formatted compiler listing 

3. The object code listing produced by the compiler 

4. The printout of the test routine’s execution 


4 , 2 Formal Testing of Release 360-4 

Release 360-4 was tested using a sub-set of the 
tests defined for 360-3 (see Appendix C) and a new 
additional set of tests. The tables and paragraphs 
below identify and summarize the purpose of these tests. 
In all, these programs represented in excess of 2000 
individual logic and language feature tests. 


preceding page blank 
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4.2.1 Sub-Set of Test Plan 


A. Discrete 

1. Integer-Scalar 

HALTEST 

OPERTEST 

EXPONTEST 

2. Vector-Matrix 

MATVECTJSOMPARE 

MATVECT_PARTITIONING 

MATVECT_EXPRESSIONS 

3. Bit 
BITTEST 

4 . Character 

CHARACTERTEST 

CHARTEST 

TEST2 

6 , Arrays 

ARRAY_ELEMENT_SELECT 

7 . Flow Control 
FLOWTEST 

B. System Support Features 

4, Compool & 5. Comsubs 

CSUB 

CPOOL 

PROG 
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General 


*D. 

1, Library & 4. Procedures 

EXPRE S S ION S_AS_ARGUMENTS 

4. Procedures 

PARM 

ECCHO 

6. Conversion 
VECFEON 

F. Special Tests 

SCRUNCH 

ED 


4.3.2 New Formal Tests for Release 360“4 

1. CONFLICT_TEST examines part of the compiler’s 
error detection facility. It is a test of the use of 
factored attributes which disagree with non-factored 
attributes in the same declaration: e.g, "Declare vector (3), 

I,J,K integer, L;", A test is also made of the compiler’s 
conflict-detection algorithm; i.e. "Declare integer, A,B, 

C integer double, D;" must not be flagged as an error. 

This category of testing (error detection) is made 
difficult by the fact that the compiler, having found one 
error, may announce further spurious errors as a result of 
incomplete error recovery. Further, the discovery of one 
error may cause the occurrence of further errors to go 
undetected. 

2 , LOG SQRT is a test of two of the built-in mathematical 
functions , It produces a table of square roots and logorithms ^ 
which may be checked ~ against any standard reference. In addition, 
the inverses of the built-in functions are used within the program 
for self checking and to determine the loss of precision involved 
in the operations. Two other tests, SIN and SINIl are essentially 
the same, except that they test the trigonometric functions. 
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3. REMEMBER_REGS tests the compiler's optimization 

of register usage* This program checks the following features: 

• 

a) That register contents are not assumed to remain 
intact across statement boundaries when a labeled 
statement or when flow control statements are 
encountered. 

b) That when a statement has too many variables for the 
number of registers available, the variables in 
registers that must be re-used are properly stored 
into memory , and 

c) That these variables which have been replaced are not 
assumed to remain in the registers in subsequent 
statements . 

4. VECTOR SHAPETEST was written specifically to insure 
that the discrepancies foxand in the vector shaping function 
at the time of Release 360-3 had been corrected. It tests 
nested shaping functions, and the use of a vector shaping 
function in the argument list to a procedure or function. 

The errors which were found in Release 360—3 were corrected. 


5. SPLAT is a test of the use of the operator ”#" to 
specify repetition in an initial or constant list in declarations. 
The program insures that nested repetition lists, such as "3# 

(14. 5, 2# (3, 4), 5)" are correctly interpreted, and that repeated 
lists with elements that require some conversion are properly 
handled. An example of this latter case is: "Declare I 

ARRAY (6) INTEGER INITIAL (2# (3, SQRT (8) , 4) ) ; " . 


6, LONG is a test of the EXIT and REPEAT statements used 
inside an iterative DO loop which has a length of more than 
4096 object bytes. The program checks that addressability is 
maintained when an EXIT or ItEPEAT causes a branch to a section 
of code which cannot be addressed from the current base register. 


7. The programs PROD, SUM, CEILING, FLOOR, MOD, and 
REMAINDER test the built-in functions of the same names. Several 
related errors were found in these library routines. For instance, 
CEILING (4.0) returned 5. This error was introduced in the single 
to double precision conversion which introduces non-zero low 

order bits, forcing the result in the next higher integer. These 
errors were corrected in Release 360-5. 
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4.3 Formal Testing of Release 360-5 and 360-6 


All previously generated tests were run using releases 
5 and 6. In addition, new test programs were written to 
augment the previous test plan. In all, by the release of 
360-6, eighty-eight test programs had been written for 
categories A, B, C, and D. These test programs contain 
approximately 8,000 individual logic and language feature 
tests . 

As a result of the tests above being conducted, nineteen 
compiler errors were discovered; 

1. Pass two of the HAL/S compiler failed to generate 
proper code for shaping functions applied to bit 
strings of length 16 arguments. 

2. One matrix exponentiation library routine was 
improperly coded, causing spurious errors and abends. 
This error was detected by MATRIX_JEXPONENTIATION. 

3. Assumed length passing in the library routine of CTOB 
caused erroneous runtime results. The error v/as 
discovered by, BSIIAPE. 

4. The use of the CSHAPE test program determined that 
improper coding in the library routine called by 
the HAL/S language form CHARACTER^j^g^ caused results 

to be proper if and only if a factor of ten was 
calculated into the original argument. 

5. The replace macro facility improperly interpreted 
macros nested within macros for the case in which a 
parameter to the outer macro was itself an argument 
of a nested macro. The error was detected by SUBBIT_ 
ON_INTANDBIT . 

6. Registers 0 and 1 were found not to be preserved in 
process swapping. Such preservation is necessary for 
the proper functioning of the code generated by pass 
two of the compiler because p^\ss two assumes that these ^ 
registers remain unchanged across statement boundaries. 
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7. Passing a bad argument to the library routine CTOI 

caused abends during runtime instead of the generation 
of the proper error message. 

•8. Character subscripting in an expression using the 

TO # subscript form caused overwriting of the calculated 
length value. Thus, erroneous results occured at run- 
time. 

9 Structure input and output caused bad code to be 

generated for the case in which the structure had a 
multiple copy specification. 

10. The length of a literal character string was incorrectly 
calculated in the compiler . 

11. During automatic template generation, multiple replaces 
in compools caused replacement of the several sub- 
sequent macro statements by the first macro string, 
regardless of the form of the later statements. 

12. Subscripts which select one copy of a multiple copy 

structure result in incorrect code: the base to which 

the calculated index is added is greater than the address 
of the first copy of the structure. 

13. The default value of the "speed" parameter was set to 

five million rather than five hundred thousand as specified. 

14. The INCLUDE function worked incorrectly when a block 
template being generated was being matched with a 
previously created template . 

15. The MAX or MIN functions did not work with arguments 
with a * array size. 

16. If two compool templates used in the same compilation unit 
possessed identically named identifiers, no error was 
detected, 

17. There was an output writer problem in printing a qualified 
structure name in a subscript. 
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18. If the dot product of two vectors of unequal length was 
taken, the Phase I error message was garbled. 

19. The multiplication of unsubscf ipted variables by argument 
less functions sometimes produced bad object code. 

20. If a program contained an identifier T then occurrences 
of the superscript form of transpose caused spurious 
cross references to appear in the symbol table entry for T 

21. In some circumstances, use of the SIZE function with 
arguments of * array size produced bad object code. 


All of these errors were corrected for Release 7 of the 
compiler. 


4,4 Formal Testing of Release 360-7 

The standard library (with additions) of formal tests 
were re-run on Release 7. Additional .tests were generated, 
covering new HAL/S language features incorporated into 
Release 360-7. A total of 4 discrepancies were reported 
during development of 360—7 , with additional testing after 
release yielding 3 more. 


4, 5 Formal Testing of Release 360-8 

The standard library (with additions) of formal tests 
was run using the release 8 version of the compiler. Additional 
tests generated for release 8 were used to test out new 
features and to add to the repertoire of previous tests. 

The new test programs are ; 

Structures : 

STRUCT_IO_COMP 

STRUCT_ACCESS 

STRUCT_ERR 

STRUCT_PARM 

Arrays: 

ARRAY_PARTITION 
ARRAY FARM 
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Replace Macros: 

REPL_TEST 

REPL_ERR 

System Language Features: 

INLN_FUNCTIONS 

TEMP_TEST 

As of the time of the release, this testing had revealed 
one discrepancy in the operation of the compiler occurring 
when a structure function used in an assignment statement. 

The final list of formal test programs is included as Appendix 
C of this document. 

Also, the precision testing of mathematical function 
subroutines in the run~time( library was begun with release 
8 of the compiler. A later memo will document results of 
this test activity. 


4 . 6 Discrepancy Reports 

Appendix D lists all reported discrepancies as of June 
30, 1974. These discrepancies were analyzed and entered 
onto the work sheets included in Appendix E. The figures 
below summarize this information. Figure 4-1 plots the 
number of discrepancies introduced into each released version 
of the compiler as a result of new code and/or the alter- 
nation of existing (good) code. The first three HAL/S-360 
releases have been grouped together and are treated as 
the original release. The drop— off in introducing errors 
is marked with only two resulting from 360-7. The trend 
is indicative of the maturity of both the compiler itself 
and programmers at Intermetrics developing it. Changes can 
now be introduced with minimum risk in causing additional 
errors. 

Figure 4-2 is interesting in that it attempts to 
convey a degree of difficulty in finding introduced errors. 
The latency value 0, 1, 2, etc,, means that the error 
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Figure 4-1 
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was introduced in one release and found in the same 
release (0), the next(l), or the next again (2), etc. 

Thus for example, 42 of the descrepancies were intro- 
duced and then discovered in the same HAL/S-360 release. 
Tl:ie average latency value is 1.1 and indicates that 
for the development so far, it takes, on the average, 
one additional release with its attendant testing, use, 
verification, etc., to find an introduced error. 

Figure 4-3 presents latency data again, this time 
with the first three HAL/S releases eliminated. Dis- 
counting this start-up "transient” it would appear that 
the total verification effort was some what better, in 
that errors were found, on the average, a little sooner. 
Of course this result might be expected as the test 
cases became more voluninous and the programmers gained 
more insight into the vulnerable areas of the compiler. 

The full story on HAL/S-360 development is, at 
this writing, incomplete. Release 360-8 has just been 
release and data will continue to be gathered. At a 
subsequent date an addendum to this report will be 
issued in which it is hoped that the reliability of the 
compiler and the effectiveness of the total test and 
verification efforts may be judged. 
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APPENDIX A 


Matrices of Tests 


<row element> means the label of the row in which an 'X* 
appears . 

<column element> means the label of he column in which an 
'X' appears. 


A.l 
A. 2 
A. 3 
A. 4 . 
A. 5 
A. 6 


List of Tables 

Assignments with Implicit Conversions 
Operator Test 
Exponentiation Test 
Unary Ope ra tor ( - ) 

Comparisons 

Initial 
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A.l Assignments with Implicit Conversions 


Lit 



Sc(s) 

4 

Sc(d) 1 

I(s) 

1(d) 

Lit (scalar) 

(integer^ 

Sc(s) 

X 

X 

X 

X 

1 ^ 

X 

Sc(d) 

; X 

X 

X 

X 

X 

IP 

X 

— l__l_ 

I(s) 

X 

X 

X 

X 

X 

W 

X 

A. 

1(d) 

X 

X 

1 ^ 

■ X 

X 

• 

X 

— 

Lit(s) 






V 

— 

Lit(I) 






■ 1 


<column element> = <row element> 


A. 2 Operator Test 



Sc (s) 

Sc(d) 1 

I(s) 

1(d) 

Lit (scalar) j 

(integer) 

1 - 

Sc (s) 

X 

1 

1 

1 


m 

f- 

Sc(d) 

X 

i- X 

1 




• 

..|t_ 

I(s) 

X 

X 

X 

\ 


i 



Kci) 

X 

X— 

X 

X 


I 

Lit (s) 

' 

X 

X 

X 

X 

X 

1 

■■ — 

Lit (I) 

X 

X 

X 

1 1_ 

X 



<column elemcnt> <operator> <row elcmont> where <operator> 

: : = t I " j <times> 1 / 
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<column eleinent> **<irow eleinent> 


An exhaustive test was necessary because the compiler makes 
a more elaborate treatment of exponentiation than the other 
operators : 

<literal>**<literal> is reduced to <literal> 
at compile time. 

I^xponentiation is done with an in-line code substitution. 
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<column element> = - <row element> 


The cases above include a few of the possible conversions. 
Conversions were tested thoroughly elsewhere; their inclusion 
here is just to test that the compiler recognizes the need 
tor a conversion. 
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A. 5 Comparisons 


Lit 



If <column element> = <row element> then 

Once one operator has been tried against all possibilities, 
the others need only one test case, The code produced will 
be similar to the following: 

CLC dependent on data types 

BC M“dependent only on operator 

Thus, a test of one operator will ensure that the correct 
comparison is done, and the test of the others is only to 
ensure that the correct mask is set up in the BC instruction. 
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A. 6 Initial 



Sc(s) 

Sc (d) 

Us) 

1 

1(d) 

jbit (scalar) 

Lit 

(integer 

1 

Sc(s) 





I ^ 


Sc (d) 





X 

X 

Us) 





X 

X 

1(d) 




• 

X 

X 

Lit (s) 







Lit (I) 








Declare <row element><attributes> initial (<column element>) ; 
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appendix B: Summary of Categories (Initial Listing) 


DISCRETE 

B.4 

GENERAL 

INTEGER-SCALAR 

VECTOR-MATRIX 

BIT 

CHARACTER 

STRUCTURE 

ARRAYS 

FLOW CONTROL 
I/O 


LIBRARY & BUILT-INS 
ARRAY-STRUCTURE INTERACTION 
RELATIONAL EXPRESSIONS 
PROCEDURE LINK & PARAMETER 
PASS 

ADDRESSABILITY & REGISTER 
USAGE 

CONVERSIONS 
SHAPING FUNCTIONS 

SYSTEM SUPPORT FEATURES 

' B. 5 

NEW FEATURES 

MACROS (REPLACE) 

INCLUDES 

FILES 

COMPOOLS 

COMSUBS 

REAL TIME 

OPTIONS IN JCL 

DEVICE DIRECTIVES 


NAME Facility 
INLINE FUNCTION BLOCKS 
PERCENT MACROS (% MACROS) 
STAND-ALONE DEBUGGING LANGUAGE 


B.3 pass 1 


ERROR CHECKS 
OUTPUT WRITER 
SCANNER 

DATA DECLARATIONS 
SCOPING RULES 
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APPENDIX B; Detailed Breakdown of Categories 


B.l DISCRETE 


Integer-Scalar 

Assignments 
Initial Values 
Comparisons 

Infix Mathematical Operators 
Exponentiation 
Prefix Operators 
Multiple Assignments 
Complex Expressions 
Complex Comparisons 

B.l. 2. Vector -Matrix 

Comparisons 
Assignments 
Initial Values 
Conversions 
Plus , Minus 

Multiply (All Types) , Divide 
Exponentiation 
Unary Operators 
Element Selection 
Partitioning 
Complex Expressions 
Complex Comparisons 
Multiple Assignments 
Built-In Functions 
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B. 1 , 3. Bit 

* Assignments 

Initial Values 
Comparisons 
Conversions 
Concatenation 
And, Or Operators 
Not Operator 
Element Selection 
Complex Expressions 
Complex Comparisons 
Multiple Assignments 
SUBBIT 

Partitioning 

Character 

Assignments 
Initial Values 
Conversions 
Comparisons 
Concatenation 
Element Selection 
Complex Expressions 
Complex Comparisons 
Multiple Assignments 
Built-In Functions 
Partitioning 
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B.1.5. Structures 


Qualified Name Reduction (assign into) 

Qualified Name Reduction (assign from) 

Template Matching 

Initialization 

Comparisons 

As Parameter or Return 
Multiple Copies 
Dense & Aligned 
Copy Selection 
Multiple Assignments 
Minor Structures 
Qualified vs. Unqualified 
Complex Subscripting 
Assignments 


B. 1. 6 Arrays > 

Element Selection 

Partitioning 

Initialization 

Mathematical Infix Operators 

Comparisons 

Parameter Passing 

Dense vs. Aligned 

Assignments 

Multiple Assignments 

Prefix Operators 

Built-In Functions 
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B.1.7 


Flow Control 


IF <bit> THEN 

IF <arrayed bit exp> THEN... 

IF <bit> THEN ELSE 

IF <bit or relational expression>- THEN 

DO... END GROUP 

Discrete DO FOR 

Iterative DO FOR 

DO UNTIL <bit> 

UNTIL <bit or relational expression>. 

DO WHILE (<bit>) 

DO WHILE (<expression>) . 

DO Case 

DO Case . . .Else 

GOTO 

EXIT 

REPEAT 

DO FOR. . .BY. . .TO 
Discrete DO FOR WHILE 
Iterative DO FOR WHILE 
Discrete DO FOR UNTIL 
Iterative DO FOR UNTIL 
DO Case <scalar> 

Nested Conditionals 
Nested DO*s 
Conversion in DO Case 
Null Statement 
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B.1.8. I/O 
» 

Implicit Positional Control 
Explicit Positional Control 
(Tabl, Skip...) 

Conversions 
Record Selection 

Null Field, <;> in read statement 
Field Width 

Generation of Device Control Characters 


B.2. SYSTEM SUPPORT FEATURES 

B.2.1. MACROS (Replace) 

Substitution of Arguments 
Nested Macros 
Scoping Conflicts 
Argument Name as Argument 

B.2.2. Includes 

Selection of Correct Data Set Member 
Text Included at Correct Point 
Tejit Introduced Unchanged 
Multiple Includes 
Include Within Include (Error) 

B.2.3. Files 

Association -of Channel Number with Data Set or Device 
Opening and Closing Files 
Standard Input File (SYSIN) 


1 
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B. 2.4 


COMPOOLS 


Template Declaration 
Compilation of COMPOOL 
Access of Contained Data 
Storing Into COMPOOL 
Access Rights 
Multiple COMPOOLS 
Locked Data 

B.2.5. COMSUBS 

Extei'nal Procedure Declaration 
External Function Declaration 
External Procedure Reference 
External Function Reference 
Procedure Compilation 
Function Compilation 

B,2,6. Real Time Control 

SCHEDULE 

TERMINATE 

WAIT 

CANCEL 

✓ 

SIGNAL 

ON ERROR 

UPDATE PRIORITY 

PRIO BUILTIN FUNCTION 

SEND ERROR 

RUNTIME 

Events ' ' 

Independent Processes 
Process Events 
Latched Events 
LocJvCd Timing 
Update Blocks 
Event Expressions 
Tasks 
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PASS 1 




Error Checks 

Set off every error message • 

Check that syntactically correct but semantically 
invalid statements produce errors. 

Check recognition of syntactically invalid forms. 

Check code production inhibition. 

B . 3 , 2 . Output Writer 

Single to multiple line conversion. 

Automatic variable annotation 
Symbol table dump 
X-ref dump 
Error summary 


B. 3. 3. Scanner 

Declaration Processing Separate Category 

Replace Processing Separate Category 

Symbol Table Entry 
Token Creation 
Multi-line Input 


B.3.4. Data Declarations 

Conflicting Attributes 
Factored Declares 
Constant Attribute 
Automatic Attribute 
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B.3.5. Scoping Rules 

‘ DECLARE VARIABLES W/ SAME NAME. 

REPLACE SEVERAL IDENTIFIERS OF THE SAME NAME IN 
DIFFERENT SCOPES. 

STRUCT TEMPLATES WITH SAME NAME IN DIFFERENT SCOPES. 
DECLARATION LOOPS 
TEMPORARY LINKAGES 
EXTREMELY DEEP NESTING 
B. 4. GENERAL 

B.4.1, Library & Built-Ins 

Matrix-Matrix 

Matrix-Vector 

Matrix-Scalar (Scalar-Matrix) 

Vector “Vector 

Vector-Matrix 

Vector-Scalar 

Scalar-Scalar 

Matrix 

Vector 

Scalar 

Integer 

Character 

Array 

No Ax^gument Functions 

Use in Expressions 
Use in Output Statement 
Use as Subscript 
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B. 4. 2. 




Assignment 
Comparison 
Multiple Assign 
Operators 


Use with mixed structure 
and array • 


Dense vs. Aligned Ascertain that conversion 

still works. 


^•4,3, Relational Expressions 

Multiple Occurrence of & , | 

Multiple Operators with Parentheses 
IF..., THEN IF... THEN IP... 

With Arrayed Relations ( = , “'=) 

V7ith Structured Comparisons 
With Function Calls 

B.4,4. Procedure Link & Parameter Pass 

Linking to External Procedures 
Passing Simple Parameters of 6 Data Types 
Passing Multiple Parameters of Mixed Type 
Assign Lists 

Linking to External Functions 

Passing Single Paraimeter of Each Data Type 

Passing Multiple Parameters 

Test Return of Six Types 
Test ’X' -length Parameter 
Implicit Conversions 
Expressions as Arguments 
Subscripted Variables in Assigns 

Use of Arrays and Structures as Parcimetexs , Assigns, 
c\nd Returns 


^.RMETRICS INCOnPORATfD • 701 CONCORD AVrNUf: • CAMRRIOGE. MASSACHUSETTS 02130 • {G17) 00 



B.4.5* 


Addressability & Register Usage 


Data 

single array > 4096 bytes 

many single elements totalling > 4096 bytes 
many structures > 4096 each 
random access indexing 

computed access where optimization applies 
Procedure 

length > 1600 instructions 

branching top-to-bottom, bottom-to-top* • . 

DO case 
DO WHILE 

long iterative DO loop 
repeat 

. large number of literals 
automatic storage 

register usage under all adverse conditions above 
B . 4 , 6 . Conversion 

a. Shaping functions covered separately except with 
single arg and radix 

b. Implicit conversions not covered under individual 
data types; 

int-bit 
bit~int 
char to int 
^char to scalar 
char to bit 
bit to char 

c. Osingle, Odouble 
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B. 4. 7 


Shaping Functions 


single argument 
arrayed arguments 
radix 

all levels of resultant subscripting 
use with argument of size 
unknown at compile-time 
nested shaping fus 
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appendix C; Test Categories & Test Programs 

(Listing current as of Release 8) 


4 

c.l. DISCRETE 

C.1.1. Integer- Sc alar 

Assignments 
Initial Values 
Comparisons 
Infix Mathematical 
Operators 
Exponentiation 
Prefix Operators 
Multiple Assignments 


Complex Expressions 
Complex Comparisons 

C.l*2. Vector-Matrix 

Comparisons 

Assignments 


Initial Values 
Conversions 

P3,us , Minus 
Multiply (all Types) 
Divide 

Exponentiation 
Unary Operators 
Element Selection 
Partitioning 
Complex Expressions 
Complex Comparisons 
Multiple Assignments 


Built-ill Functions 

C.l. 3. Bit 

Assignments 
Initial Values 
Comparisons 
Conversions 

C o nc a 1.0 1 i 1: a t i o n 

, O r Op.? r a tors 
Not Opc.'rator 


.tty 


rnTTTi 


HALTEST* 

HALTEST* 

HALTEST* 

OPERTEST* 

EXPONTEST* 

HALTEST* 

HALTEST* 

Multiple assignments to data types 
of differing type or precision will 
result in loss of precision. 

HALTEST* 

HALTEST* 


MATVECTjCOMP RE 
MATVECT_ELEMENT_SELECTION , * 
MATVECT_PARTITIONING , * 
MATVECT_EXPRESSI0NS* 

Same as above 

Implicitly tested in all Vector-Matrix 
routines . 

MIXED_TYPEJPEST 

MIXED__TYPE_TEST 

MATR IX_EXPONENTI AT ION 

MATVECT_UNARY_COMPARE* 

MATVECT_ELEMENT_SELECTION* 

MATVECT_PARTITIONING* 

MATVECT__EXPRESSIONS* 

MATVECT_UNARY__COMPARE* 

Multiple assignments to data types 
of differing precisions will result in 
loss of precision. 

MATVECT UNARY COMPARE* 


NASNTEST 

AN DTE ST,* ORTEST,* NASNTEST,* CONCTEST, 
COMPTEST 

Precision conversion implicitly 
tested in all tests, 

C0NC’r::sT 

AX!riT;ST,* ORTEST* 


KASiPi'EST* 
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* 
* -k 
■ k-k k 


Element Selection 
Complex Expressions 
Complex Comparisons 
Multiple Assignments 
SUBBIT 

Partitioning 

C.1.4. Character 

Assignments 
Initial Values 
Conversions 
Comparisons 
Concatenation 
Element Selection 
Complex Expressions 
Complex Comparisons 
Multiple Assignments 
Built-In Functions 
Partitioning 


C*1.5. Structures 

Assignments 
Comparisons 
Initial Values 
Terminal Accessing 
Element Selection 
Partitioning 
I/O 

Dense vs. Aligned 
C. 1. 6 . Arrays 

Arrayed Statements 


Subscripting 
Element_Selection 
Comparisons 
Arrayed Subscripting 

C,l,7. Flow Control 

I/O 


BITTEST* 

BITTEST* 

BITTEST* 

BITTEST* 

SUBBIT_ON_INTANDBIT 
SUBBIT_ON_SCALANDCHAR 
BITTEST,* BIT CHARACTER_SUBSCA 


CHARACTERTEST* • 
CHARACTERTEST* 

CHARTEST* 

CHARTEST,* CHARACTERTEST* 
CHARACTERTEST,* CHARTEST* 
TEST2, CINP, CH 
CHARTEST* 

CHARTEST* 

CHARTEST* 

CH, CHARTEST* 

CHARTEST,* CH, TEST2 , CINP, 
BIT CHARACTER SUBSCR 


No explicit tests 
STRUCT_IOj:OMP 
STRUCTURE TEMPLATE 
STRUCT_ACCESS 
STRUCT_ACCESS 
STRUCT_ACCESS 
STRUCT_IO_COMP 
STRUCT 10 COMP 


ARRAY_BIT__CHARJTEST 

ARRAY_PREFIX 

ARRAY AS SIGN 

ARRAY_PARTITION 

ARRAY_ELEM_SELECT 

ARRAYJ20MPARIS0N_TEST 

ARRAY_SUBSCR 

FLOWTEST*, TEST2 

Extensively Tested*** 


TEST source published in previous memo. 

See Section 3 of this report. 

Moans that features are tested by most runs during 
development c\nd use of HAL/S compiler. 
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KMUCBILOTOF^ 


C.2. SYSTEM SUPPORT FEATURES 


C.2.1. 

MACROS (Replace) 

REPL_TEST 



C.2. 2. 

Includes 

Extensively tested 
INCLUDE_TEST 



C.2.3. 

COMPOOLScompools 

All subcategories 
COMPOOL, PROG 

covered 

by CSUB, 

C.2.4. 

Comsubs 

All subcategories 
COMPOOL, PROG 

covered 

by CSUB, 

C.2.5, 

Real Time Control 

Tested by PMF Simulator 



C.3 PASS 1 

C.3.1. Error Checks 


C.3. 2. Output Writer 

C.3. 3. Scanner 

C.3. 4. Data Declarations 

Conflicting Attribute 
Factored Declares 
Constant Attribute 
Automatic Attribute 


CELT IN 

EXPONENTIATION_ERROR 

SUBBIT^ERRORJTEST 

CH 

EVENJTEST 

U NF7\CTORED_ERROR _TEST 

A S S I G N _C O N T £ X T _E RR O R 

TE PxM I N AL_S S I T_C BAR 

SUBBIT IN READ 

STRUCT_SUBBIT_IN_ASSIGN 

BLOCKSTRUCT_TEST 

NAME_PROG_ERR 

NAJ'IE^ERR 

STRUCT_ERR 

REPL_ERR 

Extensively Tested 
Extensively Tested 


CONFLICT^TEST 

Implicitly tested in many routines’ 

CONSTANT_TEST 

Development Tests 


C.3. 5, Scoping Rules 


Implicitly tested 
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C.4 


GENERAL 


C.4.1, 


Library and Built-Ins 
Integer & Scalar 


COS_TEST, COSH_TEST, LOG_SORT 

Values of results have been compare 
with a CRC tables and were found to be 
correct within the accuracy of the 
tables 




Character 

Array 

No Argument Functions 
Use in Expressions 

Use in Output 
Statement 
Use in Subscript 


MAXIMUMPRODSUM 

MODTEST 

SIGN_SIGNUM_TEST 

DIV ANDRH-IAIN DERTE ST 

CEILINGFLOOR_TEST 

EXPRESS ION_AS_ARGUMENTS 

CH, CINP, CHAi^CTERTEST* , CBLTIN 

SIZE_TEST 

Only RUNTIME Explicitly tested 
Tested in the complex expressions 
subcategory of each category 

Development Tests 

Tested as part of the general sub- 
scripting test for each datatype 


C . 4 . 2 . Array- Structure 
Interaction 


Development Tests 


C.4. 3. 


Relational 

Expressions 


Subcategories covered by parts of 
FLOWTEST, HALTEST, CHARTEST 
BITTEST, MATVECTjCOMPARE, 
ARRAYCOMP ARE I SONTEST 


C.4. 4. Procedure Link & 
Parameter Pass 

Linking to External 
Procedures 

Passing Simple 

Parameters of 6 
Data Types 
Passing Multiple 
Paran^eters of 6 
Data Types 
Assign Lists 
Linking to I-b:ternal 
Functions 


CSUB,‘ PROG 

Development Tests 

PA^'L T4, ECCIIO 

FARM, i:CCiiO, T3 

Extensively tested by both FDI 
a n d PM i ' • ' -i i • i u 1 a t or s * 


* Sec Sccti.on 3 of this report. 


Ik* n t m\ 11 -T 1 CS I Nc:0 Ri ’ORA 1 1.0 


50 

voi CO MOO 1^0 avlnui: 


l:a^‘ 


■ Cl lusf 1 1 s o:’ 1 33 • (o I ■/) or. i - 1 o-i o 



^ Passing Single Para- 
meter of Each Data 
Type 

Passing Multiple 
Parameters 
Test Return of Six 
Types 

Test '* '-length 
Parameter 

Implicit Conversions 
Expressions as 
Arguments 

Subscripted Varia- 
bles in Assigns 
Use of Arrays and 
Structures as 
Parameters, Assign: 
and Returns 

C . 4 , 5 . Addressability -and 
Register Usage 

C . 4 . 6 - Conversion 

Integer-Scalar 

Shaping Functions 
Vector -Matrix 

Shaping Functions 

C.4.7. Shaping Functions 
SUBBIT 

Bit Shaping Functions 
Character Shaping 
Functions 

C.5. NEW FEATURES 


Tested in the complex expressions 
subcategory of each datatype 

PARM, T4 

Tested in the complex expressions 
subcategory of each datatype 

SIZE_TEST 

Used in all tests 

EXP RES S IONS_AS_ARGUMENTS 

SUBSCRIPT_ASSIGN_PARTITION 

; , STRUCT__PARM 
ABRAY__PARM 

Extensively Tested 

INTEGER_SCALAR_SHAPE, ISSIiAPEjDN_MISC 
VECFUN, VECTORSIIAPEJTEST, MSHAPE 

SUBB IT_ON_SCALANDCHAR 
SU BB I T_ON_I NTARDB IT 
BSHAPE 

CSHAPE 


C . 5 . 1 . System HAIy 


NAME 

Rii: PLACE :‘L\CROS 
t MACROS 
TEMPORARY 
IN-LINE FUNCTIONS 


NAME__PROG 

REPLACE_BUGs* 

Dcvo I o:>;.iCn ta. 1 Tests 

TEMP_TEST 

INLIN FUNCTIONS 


Let of routine:': 
etc . 


to t e St s t a c k i i uj , p u. i; to r s , 


macro 


inside-m.icro;: , 
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C.5.2. Error Library 


An extensive error check library has been developed. 
Some programs in this library are designed to trigger the 
error message and error recovery mechanisms v/ithin the 
compiler. These tests verify whether the appropriate 
error messages are being generated and whether the error 
recovery mechanism responds in a fashion which allows 
subsequent statements within the compilation to be 
processed despite the fact that no code will be generated 
by the particular run. This allows the user to review 
the majority of his current syntactic or semantic errors 
before resubmitting the job. Other programs in the error 
check library are designed to trigger runtime errors. 
These tests verify the proper generation of an error 
message, and the possible trace information associated 
with the message. In addition they check the GO TO, 
IGNORE, and SYSTEM options available with the ON ERROR 
statement. 


C.6. SPECIAL TESTS • INT TO_BIT, ED, QUEUES, PARENTHESIS, 

CHECKER, DEBUG, SCRUNCH, ARITH, 

TESTGEN, BTESTGEN, KIP, FORMAT, 

PMSPOOL, FDA 

• FDI-Simulator 

• 2 Compools 

. 13 COMSUBS 

• 39 internal functions and procedures 
Over 2000 HAL/S statements 

PMF ~ S imu 1 a tor 


real time 


1 Compool 
4 programs ) 

8 tasks / 

20 internal functions and procedure 
Over 2 000 IIAL/S statements 


Math Function Precision Tests 
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APPENDIX D 


HAL/S Discrepancy Report Log 
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Fixed 


Source I 

1 Title 

1 

intermetrics 

•Date 

for 


Date Rcc. 

Evaluation 

'Fixed 

Release 

1 

( 

csdL 

1 

9/05/73 

! 

i 

Replace Text 
Identifiers 

confirmed bug 

8/15/73 

360-3A 

CSDL 

9/05/73 

! Initialization of 
Integers 

•t 

8/15/73 

360-3A 

CSDL 

9/05/73 

t 

1 Bit Function 
i Writes 

rt 

8/15/73 

360-3A . 

RI 

9/17/73 

Character 

Initialization 

n 

9/05/73 

360-4 

CSDL 

1 9/18/73 

i 

1 

Trailing Under-* 
scores in identi- 
fiers 


9/25/73 

360-4 

CSDL 

9/13/73 

END Labels 

n 

9/01/73 

360-4 . 

CSDL 

9/18/73 

READ Statements 

n 

1 

9/27/73 1 

360-4 

IBM 

10/04/73 

Half-word sign bit 

ft . 

10/04/73 ' 

,360-5 

IBM 

10/04/73 

Bit catenation 

II 

10/04/73 

360-5 

CSDL 

10/10/73 

Replace Statements 

If 

10/14/73 

360-5 

CSDL 

10/18/73 

j Precedence. Inconsis 

1 

If 

10/20/73 

360-5 

RI 

1 

10/24/73 

1 

} Arith. Functions 

i 

II 

- 

360-4 

RI I 

i \ 

10/24/73 

1 

1 DO END Label ! 

See DR #6. 



IBM 

10/29/73, 

1 

I Matrix Shape Error j 

Not -.available 

in 360-4 









cn 

Ui 


HAL/S DISCREPANCY REPORTS 


__ — 

Source 

Title 

Intermetrics 

Date 

Fixed 

for 

NuHiber 

V.’ho? 

Date ’Pvec« 

Evaluation 

'Fixed 

Release 

15 

hiiM 

1 

10/29/73 

IF Statement 

Not a bug 



16 

' IBM, 

10/29/73 

Error cleanup after 
INITIAL 

Confirmed bug 

10/29/73 

360-5 

17 

IBM 

10/29/73 

END Lables 

See DR #6 

• • ■ 


18 

IBM 

10/29/73 

EOF Looping 

Confirmed bug 


360-5 

19 

IBM 

10/29/73 

Comparison failure 

11 


360-5 

20 

IBM 

10/29/73 

ARCCOS Results 

11 1 


360-5 

21 


10/08/73 

Argument Count 

It 

10/12/73 

360-5 

22 

,2 

± 

10/08/73 

Array Loop Gener. 

II 

10/12/73 

360-5 

23 


10/12/73 

Literal, in Built-in 

M 

10/12/73 

360-5 

24 


10/15/73 

Init* of Bit Var. 

ft 

10/17/73 

360-5 

25 

l2 

10/17/73 

Use of RO 

II 

10/17/73 

360-5 

26 

t2 

10/20/73 

Shaping Functions 

t1 

10/28/73 

360-5 

27 


10/28/73 

Single Precision 
Literals 

» 

10/28/73 

360-5 

28 

IBM 

10/22/73 

V7riter Formatting 
Error 

M 

11/1/73 

360-5 

29 

IBM 

10/29/73 

INITIAL/AUTOMATIC 
on Assign Var» 

tt 

11/1/73 

360-5 

30 

IBM 

10/30/73 

ARCSIN/ARCTAN 

problems 

M 

11/11/73 

360-5 


Comitient 


EEPRODUCmiLITY OF THE 
ORIGINAL PAGE IS POOR 



HAL/S DISCREPANCY REPORTS 




Source 


Intermetrics 

Date 

'Fixed 

Fixed 

for 

Release 

Humber j 

VJho? 

Date Rec. 

Title 

Evaluation 

- 

31 


\ IBM ' 

10/31/73 

ELEMENT Selection 
from bit string ' 

Confirmed bug 

11/6/73 

360-5 

32 


IBM 

10/31/73 

Bit Partitioning 

II 

11/6/73 

360-5 

33 


IBM 

11/1/73 

Bit Partitioning 
and # 

« .... 

11/3/73 ■ 

360-5 

34 


CSDL 

11/3/73 

Inconsistency in 
Array Compares 

-See DR 
#22 ■ 



35 



11/3/73 

Structure Assign 
Farms 

Confirmed bug 

10/6/73 

360-5 

36 


IBM 

11/13/73 

IF/ELSE Indentatioi 

« 

11/13/73 

360-5 

37 


IBM 

11/13/73 

Invalid Opcode 
Generation . 

It 

• 

11/10/73 

360-5 

38 


IBM 

11/28/73 

Illegal FP Reg. 
Alloication 

Side effect of 
#39 



39 


IBM 

11/28/73 

Invalid Opcode 
Generation 

Confirmed bug 

11/15/73 

360-5 

40 


d 

11/28/73 

NONHAL Procs & Punc 

s 

11/28/73 

360-6 

41 


d 

11/28/73 

Array Args of 
Functions 

II 

12/1/73 

360-6 

42 



11/27/73 

Dynamic FP Range 

tt 

11/28/73 

360-6 

43 

*4 

l2 

11/29/73 

Arrayed Statement 
as case of DO CASE 

tt 

12/1/73 

360-6 


Comment 



Number 
4 4 

45 

46 

47 

48 

Ln 

•>4 

49 

50 , 

51 

52 

53 

54 

55 



Source 


Date Rec. 


Repeat Group in 
Shaping 

conversion 



RAL/S DISCREPANCY REPORTS 


Title 


rted b 


Intermetrics 

Evaluation 

Date 

’Fixed 

Fixed 

for 

Release 

Confirmed bug 

12/4/73 

360-6 

. tl 

> 

12/7/73 

360-6 

11 

t 

12/7/75 ‘ 

360-6 

ir 

12/8/73 

360-6 

Not Error 



Confirmed 

12/20/73 

360-6 

Not Error 



Confirmed 

L2/20/73 

360-6 

n 

1/2/74 

360-6 

3 » 

12/20/73 

360-6 

ir 

12/19/73 

360-6 

ti 

12/19/73 

360-6 


Comment 


Documentation 

Problem 


REPRODUCIBILrry OF THE 
ORIGINAL PAGE IS POOR 













HAL/S DISCREPANCY REPORTS 




1 

— 



Fixed 




Source 1 


Intermetrics 

Date 

for 



NuTTibcr 

Who? 

Date 'Rec. 

Txti.e 

Evaluation 

'Fixed 

Release 


56 


12/19/73 

Structure I/O 

Confirmed 

12/19/73 

360-6 



57 


12/21/73 

Replace Text in 
Templates 

II 

12/21/73 

360-6 



58 

RI 

12/20/73 

lExceptions -Output 
(Writer with REPLACE 


• ■ • 




59 


1/02/74 

Bits in Range 9-16 
bits 

Confirmed 

1/02/74 

360-6 



60 

l2 

1/02/74 

Bit(16) Vars. and 
Assignments 

It 

1/02/74 

360-6 



61 


1/02/74 

Structure Parame.ters 

•1 

1 

1/02/74 

360-6 



62 

CSDL 

12/09/73 

Precision- 

H 1 

1 

1/02/74 

360-6 



63 

CSDL 

12/09/73 

Addition Failure' 

Confirmed 



eliminated 

in re- 

64 

CSDL 

12/09/73 

Array I/O 

Part Confirmed 


- r 

release of 

360-6 

65 

CSDL 

12/09/73 

Macro-text Listings | 

Confirmed 

12/10/73 

360-6 



66 

CSDL 

12/09/73 

Vector Shaping 

Confirmed 

12/10/73 

360-6 



67 

CSDL 

12/09/73 

Overflow : 

See DR #46 





63 

CSDL 

12/09/73 

Overflow(2) 

Confirmed 





69 

CSDL 

12/09/73 

Character 0 DEC 

Confirmed 

12/20/73 

360-6 



70 

CSDL 

11/22/73 

Procedure Parmeters 

Confirmed 

1/15/74 

360-6 



71 

CSDL 

11/24/73 

Arrayness Conflict 

Confirmed 

1/15/74 

360-6 



72 

CSDL 

11/20/73 

Link Step OC5 

Confirmed 

1/15/74 

360-6 



73 

CSDL 

11/20/73 

Phase II Error in 
Arrayed Args. 

Part Confirmed 

1/15/74 

360-6 

• 


74 

CSDL 

11/24/73 

Indirect Stack Over- 

-P T , 

' Confirmed 

1/15/74 

360-6 





hal/s DISCREPANC'/ reports 




1 r 



• 

Fixed 



Source 


Intermetrics 

Evaluation 

Date 

for 

Comment 




Title 

. Fixed 

Release 

I^jir.bcr 

v:ho7 I 

;)ate Rec. 

“ — — 1 

75 

CSDL ' 

11/16/73 

Addressability ( 

Compiler Restrictio: 

n 1/15/74 

360-6 

1 

1 

7G 

\ 

csih 

12/7/73 

Args of NONHAL 

Not Error 






Procedures 





77 

CSDL 

12/21/73 

Rounding 

Confirmed 

1/10/74 

360-6 


78 

CSDL 

1/16/74 

SUBBIT pseudo- 
variable 

Confirmed 

1/30/74 

360-6 


79 

I3M ! 

12/10/73 

Abend 1/2 word BIT 
OPS 

> See DR #59, 6 : 

1 



30 

i IBM 

12/12/73 

1/2 word bit compai 


• . 



81 

! 

IBM 

1/8/74 

Variable Dump of 
Constraints 

Not Error 




82 

IBM 

11/16/73 

Statement # Trace 

Not Error 




83 

IBM 

11/19/73 

Trace Diagnostic 
Problem 

? 


1 

Trace Package 
changed. Bug non- 

existent in Ver-* ■ 
Sion 5. 

84 

:SDL 

1/29/74 

Satalog Procedure 

Confirmed 


360-6 



Error 





85 . 

CSDL 

1/29/74 

Execution with 
Compiler Error 

Confirmed 

1/30/74 

360-6 • 


86 

CSDL 

1/29/74 

S&M’s split over 

Confirmed 

1/30/74 

360-6 




page 





87 

CSDL 

1/30/74 

Block Summaries 

Confirmed 

1/30/74 

360-6 


88 

CSDL 

1/29/74 

Failure of MAX-MIN 

Confirmed 

3/3/74 

360-7 


89 , 

CSDL 

1/29/74 

! 

1 

RG9 Message Wordin 

,t[ Confirmed 

1/30/74 

360-6 

■ 

ft 



hal/s discrepancy reports 




r 




Fixed 



Source 

Title 

_ Intermetrics 

Date 

for 

Comment 

* 



Fixed 

Release 


v:ho? ! 

Date Rec* 

£* V a iuo. uXQIi 





90 ■ ' 

csc^ 

( 

1/29/74 t 

Jnlatched Event 
Initialization 

Not Error 


1 

/ documentation 
{ problems 

91 ' 

CSDL 

1/29/74 • < 

ODD Function 

Not Error 




92 

CSDL 

1/29/74 

Compile time OC5 

‘Not Error 






Dumps 





93 

CSDL 

2/12/74 

Random Error Message 

js Not Error 




94 

CSDL 

2/18/74 

Writing Structures 

Confirmed 

1/30/74 

360-6 


95 

CSDL 

2/18/74 

Indexing Error 

Confirmed 

1/30/74 

360-6 


. 96 . 

CSDL ' 

2/18/74 

Double Explicit . 

Confirmed ■■ 

2/12/74 

360-6 




Conversion 





97 

CSDL 

2/13/74 

Unlatched Event 
Initialization 

See DR #90 



1 

98 

CSDL 

2/18/74 

Inter-Process, ■■ * 

Not Error 






Boolean 





99 

CSDL 

2/18/74 

Multiply Defined 
Compools vars. 

confirmed 

1/30/74 

360-7 


I'OO 


3/5/74 

INTEGER/SCALAR 

shaping 

Confirmed 

3/12/74 

360-7 



CSDL 

3/20/74 

Functions as 

Confirmed 

3/15/74 

360-7 


101 

Multipliers 




102 

CSDL 

3/20/74 

Vector Length in 

Confirmed 

3/15/74 

360-7 


1 

dot product 






1 ^ 3 / 2 ^ 

1 

1 

in i^in<4 


3/15/74 

360-7 

I *• • 

f 



hal/s discrepancy reports 



Source 


- Interinetrics 

Date 

Fixed 

for 

Comment 





Title 

Eva luation 

Fixed 

Release 


Niimbcr 

V7ho? h 

3ate Rec. 




104 

CSDL 

3/20/74 1 

Inlines in Replaces 

Confirmed 

3/15/74 

360-7 



LC5 


3/5/74 

argument of SIZE 

Confirmed 

3/12/74 

360-7 



105 


3/7/74 1 

JPDATE BLOCKS & DO 

Confirmed 

3/15/74 

360-7 . 




CASE 






107 


3/7/74 C 

Output Writer Sub- 
script Problem 

Confirmed 

\ 





103 

15DL 

3/20/74 

Symbol Table Loop 

Confirmed 

3/18/74 

360-7 


S 3 
2 S 

S 2 

1C9 

CSDL 

3/20/74 

DECLARE Tasks 

Confirmed 

4/08/74 

360-8 

LCR Required 

110 

; CSDL 

3/20/74 

Single Free. Ex-r | 

Confirmed 




i 


plicit Conversion 

See DR #106 • 




• o 

111 • 

CSDL 

3/20/74 

DO CASE Error 


1 



112 

CSDL 

3/27/74 

^AL/S Initializatior 

Confirmed | 

3/25/74 

360-7 


M 1^ 

113 

CSDL 

3/27/74 

READ-ALL Structurej 

Confirmed 

3/25/74 

360-7 


o 

114 

CSDL 

3/27/74 

READ-ALL Skip 

Confirmed 

3/25/74 

360-7 



115 

CSDL 

3/29/74 

Infinite Loop in 

Confirmed 

3/25/74 

360-7 


'"S 


GO TO 






116 

CSDL 

4/05/74 

Init. of Var, with 

Confirmed 


360-8 





NULL- String 






117 

IBM 

4/24/74 

Garbage in Macro 

Confirmed 


360-8 





Expansion 






113 > 

IBM 

4/24/74 

Precision of Mixed 

i Confirmed 


360-8 ■ 




1 i 

Free. Divide 


, 






hal/s discrepancy reports 








Fixed 



Source 

Title 

■ Intermetrics 

Date ■ 
Fixed 

for . 
Release 

Comment 

Number 

Who? i 

Date Rec. 

^ 1 A JL O 

119 

1 

4/18/74 

Range of Error 
Numbers 

Confirmed 


360-8 


120 


4/24/74 

Source Macro's & 
Templates 

Confirmed 


360-8 


121 

l2 

5/02/74 

Transpose Problem 



360-8 ■ 


122 

CSDL 

5/02/74 

Nested SUBBITS 

Spec, Problem 




123 

CSDL 

5/02/74 

AP-101 Timing 
Anomalies 

. Confirmed 


360-8 

1 


124 

CSDL 

5/02/74 1 

DEVICE Directive | 

Confirmed 


. 360-8 


* 125 ' 

CSDL 

5/13/74 

Assignment of BIT 
Literals 

Confirmed 

5/19/74 

360-8 


126 

CSDL 

5/13/74 

Reading Structures 

Conf irmed 

5/19/74 

360-8 


127 

CSDL 

5/13/74 

Size Function 

Confirmed 

5/19/74 

360-8 


128 

CSDL 

5/13/74 

Var. names length 
32 

Confirmed 

5/19/74 

360-8 


f — i 


6/1/74 

Phase I abend in 
macro expan . 

Confirmed 

5/12/74 

360-8 ■ 


130 


6/1/74 

Replace macros in 
symbol table 

Confirmed 

6/12/74 

360-8 


131 


6/1/74 

Literals & Constant 

:■ Confirmed 

6/12/74 

360-8 


132 


5/16/74 

Names of programs 

Confirmed 

•6/5/74 

360-8 



! 


& tasks 







HAL/S DISCREPANCY REPORTS 



cn 

Ou) 


Who? 

IBM 

IBM 

IBM 

IBM 

I 



Date Rec . 


Title 


5/15/74 Simulation Vector 
• Table 


6/1/74 OCI Abend in 02 

6/12/74 Bad code for DO CASE 

3/1/74 , Bad printout' of struc 

ture terminals 


6/19/74 ’<label>* for ' C’ 



Intermetrics I Date 
Evaluation | "Fixed 

ICD Discrepancy 

Confirmed 6/12/74 

•• 6/19/74 

" 6/19/74 


Fixed 

for 

Release * Comment 

360-8 
360-8 

360-8 











Appendix E 


REPRODUCIBILITY OP THE) 


Discreoancv Worksheet ORIGINAL PAGE IS POOR 



PRECEDING PAGE BLANK NOT FILMED 


Legend: 

X; 


/: introduced 

0: found in 


65 


fixed in 








































66 































reproducibility op Tffl 
ORIGINAL PAGE IS POOR 













































68 






















REPRODUCIBILnY OF THE 
ORIGINAL PAGE IS POOR 



69 

























